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Abstract 

This article is the starting point of a series of works whose aim is the study of deterministic 
control problems where the dynamic and the running cost can be completely different in two (or 
more) complementary domains of the space M^. As a consequence, the dynamic and running 
cost present discontinuities at the boundary of these domains and this is the main difficulty of 
f~| ■ this type of problems. We address these questions by using a Bellman approach: our aim is 

\ to investigate how to define properly the value function(s), to deduce what is (are) the right 

Bellman Equation(s) associated to this problem (in particular what are the conditions on the set 
where the dynamic and running cost are discontinuous) and to study the uniqueness properties 
for this Bellman equation. In this work, we provide rather complete answers to these questions 
\ in the case of a simple geometry, namely when we only consider two different domains which 

■ are half spaces: we properly define the control problem, identify the different conditions on 
I the hyperplane where the dynamic and the running cost are discontinuous and discuss the 

■ uniqueness properties of the Bellman problem by either providing explicitly the minimal and 
CO I maximal solution or by giving explicit conditions to have uniqueness. 

(N; 
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X ; 1 Introduction 



In this paper, we consider infinite horizon control problems where we have different dynamics and 
running costs in the half-spaces Q,i := {.x € : xj\[ > 0} and Q.2 '■= {x G : < 0}. 

On each domain (i = 1,2), we have a controlled dynamic given by 6j : x — > M^, 
where Ai is the compact metric space where the control takes its values and a running cost li : 
QiX Ai ^ R. We assume that these dynamics and running costs satisfy standard assumptions: the 
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functions Oj), Oi) are continuous and uniformly bounded and the Oi) are equi-Lipschitz 
continuous. To simplify the exposure, we also suppose that the system is controllable on both sides. 

The first difficulty is to define the controlled dynamic and in particular for trajectories which 
may stay for a while on the hyperplane := PI O2 = {x G : xn = O}. To do so, we follow 
the pioneering work of Filippov |17j and use the approach through differential inclusions. As a 
consequence, we see that in particular there exist trajectories which stay on "H at least for a while. 
Such trajectories are build through a dynamic of the form 

bnix, (ai,a2,/^)) := fJ.bi{x,ai) + (1 - /u)62(a;,a2) , 

for X € 7i, with /i € [0,1], € Ai and 6^(x, (ai, 02, A*)) ■ cat = where '■= (0, • • • ,0,1). We 
denote by Ao(x) the set of such controls a := (ai,a2,//). The associated cost is 

Inix, a) = Inix, (ai, 02, /")) := fJ-hix, ai) + (1 - fJ-)l2ix, 02) . 

Once this is done, we can define value-functions and look for the natural Bellman problem(s) which 
are satisfied by these value functions. Actually we are going to define two value functions, we come 
back on this point later on. 

It is well-known that, for classical infinite horizon problems, i.e. here in Qi and ^2) the equations 
can be written as 

Hi{x,u,Du)=0 in , , , 

H2{x,u,Du) = in 1^2, 

where Hi , H2 are the classical Hamiltonians 

Hi{x,u,p) := sup {-bi{x,ai) ■p + Xu-li{x,ai)} , (1.2) 

where A > is the actualization factor. From viscosity solutions' theory, it is natural to think that 
we have to complement these equations by 

mm{Hi{x,u, Du), H2{x,u, Du)} < onH, (1-3) 

max{Hi{x,u, Du), H2{x,u, Du)} > onT-L. (1.4) 

This is actually true since the two value functions we introduce naturally satisfy such inequalities. 
Note that, for the sake of simplicity, we always say a sub and supersolution of (ll.ip -( fL3l )- (jl.4p . 
while it has to be understood that both verify (|1.1|) in 0,i and ^^2) but a subsolution only satisfies 
(jl.3p on Ti while a supersolution only satisfies (jl.4p on Ti. 

The main interesting questions are then 

1) Does problem (jl.ip - (jl.3p - ()1.4p have a unique solution ? 

2) Do the value functions satisfy other properties on T-L 7 

3) Do these extra properties allow to characterize each of the value function either as the unique 
solution of a Bellman problem or at least as the minimal supersolution or the maximal sub- 
solution of them? 
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Our results give complete answers to the above questions. 

Concerning Question 1), the answer is no in general. We do not have uniqueness for the prob- 
lem (jl.ip - (jl.3p - ()1.4p but we can identify the maximal subsolution (and solution) and the minimal 
supersolution (and solution) of (|l.ip - (|1.3p - (jl.4p : they are value functions of suitable control prob- 
lems which we are going to define now. The difference between the two is related to the possibility 
of accepting or rejecting some strategies on Ti. To be more precise, we call singular a dynamic 
b-H{x, a) on Ti (i.e. such that bfc{x, a) ■ e]\f = ) when bi{x, ai) • bat > and b2{x, 02) • cat < while 
the non-singular (or regular) ones are those for which the 6j(x, ai) ■ have the opposite (may be 
non strict) signs. Then, the minimal solution U~ is obtained when allowing all kind of controlled 
strategies (with singular and regular dynamics) while the maximal solution U'^ is obtained by 
forbidding singular dynamics. The uniqueness problem comes from the fact that, in some sense, 
the singular strategies are not encoded in the equations (ll.ip - (jl.3[) - (11.4p . while it is the case for the 
regular ones. 

For Question 2), the answer is the following: if we allow any kind of controlled strategies, both 
the regular and the singular ones, the associated value function, namely U~, also satisfies the 
inequality 

Ht{x,u,D'hu) <0 onn, (1.5) 

where -D-^u := (^^, • • • , -g§^^) is the gradient of u with respect to the ^^-variables xi, • • • ,Xn-i 
and, for x £ 'H,u £ M.,p' ^ Ht{x,u,p') is given by 

sup {-6-^(3;, o) • (/,0) + Xu- ln{x,a)} . 

Ao{x) 

We emphasize the fact that this viscosity inequality is actually a M''^"^ viscosity inequality (meaning 
that we are considering maximum points relatively to T-L and not to M^); it reflects the suboptimality 
of the controlled trajectories which stay on T-L. This inequality makes a difference between U~ and 
U"'' since U"*" satisfies the same inequality but with Ao(x) being replaced by A^q^{x) consisting in 
elements of Aq{x) satisfying bi{x, ai) • bat < and b2{x, 02) • cat > 0. 

For Question 3), (II. 5p also makes a difference since there exists a unique solution of (|l.ip - (|1.3p - 
(jl.4p - (jl.5p . In other words, the uniqueness gap for (jl.ip - (jl.3p - (jl.4p just comes from the fact that 
a subsolution of (ll.l|) - (ll.3|) - (ll.4p does not necessarily satisfy (jl.Sp and this is due to the difficulty 
to take into account (at the equation level) some singular strategies. We illustrate this fact by an 
explicit example in dimension 1. 

Besides of the answers to these three questions, we provide the complete structure of solutions 
in 1-D and we also study the convergence of natural approximations. 

We end by remarking that there are rather few articles on the same topic, at least if we insist 
on having such a structure with a general discontinuous dynamic. A pioneering work is the one 
of Dupuis [16] that considers a similar method to construct a numerical method for a calculus of 
variation problem with discontinuous integrand. The work of Bressan and Hong [Tl] goes in the 
same direction by studying an optimal control problem on stratified domains. Problems with a 
discontinuous running cost were addressed by either Garavello and Soravia [181 119] , or Camilli and 
Siconolfi [12] (even in an L°°-framework) and Soravia [27]. To the best of our knowledge, all the 
uniqueness results use a special structure of the discontinuities as in [lU [HI [20] or an hyperbolic 
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approach as in [31 [13] . We finally remark that problems on network (see [2l] , [2] , [26] ) share the 
same kind of difficulties. 

The paper is organized as follows: in Section [2l we show how to define the dynamic and cost of 
the control problem in a proper way, we introduce two different value functions (U~ and U"*") and, 
in Theorem 12.51 we show that they are solutions of (jl.ip - (jl.3p - (jl.4p . In addition, we prove that U~ 
satisfies the subsolution inequality (jl.Sp while U"'' satisfies a less restrictive inequality, associated 
to the Hamiltonian involving only regular controls < Ht- Section [3] is devoted to study 

the properties of any sub and supersolution of (|1.3p - (|1.4p - (|1.5p and, in particular, the additional 
inequalities that they satisfy on 7i (inequalities which are connected to Hj- or H^^). In Section [31 
we use these properties to provide a comparison result for (jl.ip - ([TT^ - (|1.4p - (|1.5p (Theorem 14. ip : one 
of the main consequences of this result is that U" is the minimal supersolution and solution of (jl.ip - 
p^ - ([L3|) . while U+ is the maximal subsolution and solution of ([LT]) - ([L3|) - p^ (cf. CoroUary[331)- 
In Section [5l we study in details the case of the dimension 1 by providing the complete structure 
of the solutions, together with examples of different behaviors. Finally Section [6l is devoted to 
examine the effect of several approximations (Filippov and vanishing viscosity). 

2 A control problem 

The aim of this section is to give a sense to infinite horizon control problems which have different 
dynamic and cost in Qi := {x G : xn > 0} and in 0,2 ■= {x € : xn < 0}. Of course, the 
difficulty is to understand how to define the problem on H := H Q2 = {x G : xn = O). 

We first describe the assumptions on the dynamic and cost in each (i = 1,2). On Qi, the 
sets of controls are denoted by Ai, the system is driven by a dynamic 6j and the cost is given by li. 

Our main assumptions are the following 

[HO] For i = 1, 2, is a compact metric space and 6j : x Ai ^ is a continuous bounded 
function. Moreover there exists Lj € R such that, for any x,y & R^ and ai € Ai 

\bi{x,ai) - bi{y,ai)\ < Li\x - y\ . 
[HI] For 2 = 1,2, the function li : R^ x Ai ^ R^ is a continuous, bounded function. 

[H2] For each x G R^, the sets {{bi{x,ai),li{x,ai)) : ai G Ai}, [i = 1,2), are closed and convex. 
Moreover there is a 5 > such that for any i = 1,2 and x € R^, 

B{Ojj C Bi{x) := {6,(x, ai) : a, € Ai} . (2.1) 

Assumptions [HO], [HI] are the classical hypotheses used in infinite horizon control problems. 
We have strengthened them in [H2] in order to keep concentrated in the main issues of the problem. 
Indeed, the first part of assumption [H2] avoids the use of relaxed controls, while the second part 
is a controllability assumption which will lead us to Lipschitz continuous value functions. In a 
forthcoming work, we are going to weaken [H2] by assuming only some kind of controlability in the 
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normal direction: this weaker assumption is inspired from [8] where it is used to obtain comparison 
results for discontinuous sub and super-solutions of exit time-Dirichlet problems without assuming 
the "cone's condition" of Soner In our framework, H plays a similar role as the boundary 

of the domain in exit time problems since one of the main question is how the trajectories of the 
dynamics reach "H and how the value function behaves on H. In [27], Soravia uses a transversality 
condition which looks like the "cone's condition" of Soner [25] to prove comparison results while 
[H2] or its weaker version are more related to the Barles-Perthame approach [8] (See also [5]). 

In order to define the optimal control problem in all M^, we first have to define the dynamic 
and therefore we are led to consider an ordinary differential equation with discontinuous right-hand 
side. This kind of ode has been treated for the first time in the pioneering work of Filippov [IT]. We 
are going to define the trajectories of our optimal control problem by using the approach through 
differential inclusions which is rather convenient here. This approach has been introduced in [28] 
(see also [1]) and has become now classical. To do so in a more general setting, and since the 
controllability condition (j2.ip plays no role in the definition of the dynamic, we are going to use 
Assumption [H2]„c which is [H2] without (j2.ip . 

Our trajectories Xxq{-) = (^Xx^^i, Xx(^^2, ■ ■ ■ ,Xxq^n){-) are Lipschitz continuous functions which 
are solutions of the following differential inclusion 

X^o (t) e B{Xx, (t)) for a.e. t G (0, +oo) ; X^, (0) = xq (2.2) 

where 

Bi{x) if X]\f > , 

B{x) := < B2{x) if3;Ar<0, 

_ co(-Bi(x) U-B2(x)) ifXAr = 0, 

the notation co{E) referring to the convex closure of the set E C M^. We point out that if the 
definition of B{x) is natural if either xat > or xat < 0, it is dictated by the assumptions to obtain 
the existence of a solution to (j2.2p for xjy = (see below). 

In the sequel, we use the set A := Ai x A2>^ [0, 1] where the control function really takes values 
and we set A := L°°(0, -|-oo; A). We have the following 

Theorem 2.1. Assume [HO], [HI] and [H2]nc- Then 

(i) For each xq € M^, there exists a Lipschitz function Xxq : [0, oo[— )■ which is a solution of the 
differential inclusion ()2.2p . 

(a) For each solution Xxq{-) of (j2.2p . there exists a control a(-) = [ai{-),a2{-), fJ^i-)) € A such that 
Xxoit) = ^i(^^o(*)>ai(*))l|x.„wef7i} + ^2(^x0 (i),«2(i))l|x. jt)gn2} 

(2.3) 

+ 6^(X,,(t),a(t))l|^^^(,)g^| for a.e. t G M+ , 

(where Ia(') stands for the indicator function of the set A.) 
(Hi) // Cat = (0, • • • , 0, 1), then 

bn{Xxoit),a{t)) • CAT = a.e. on {X^o^Nit) = 0} . 
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Proof. This result follows from two classical results in [T]. 

Step 1 - Since the set-valued map B is upper semi-continuous with convex compact images, thanks 
to [H Theorem 10.1.3], we have that, for each xq G M^, there exists an absolutely continuous 
solution Xxq{-), of the differential inclusion (j2.2p . i.e. 

Xx, (t) G B{Xx, [t]) for a.e. t G (0, oo) ; (0) = xq . 

Note that the solution is defined in all and Lipschitz continuous, thanks to the boundedness of 
B. This first step justifies the definition of B for X]\f = 0. 

Step 2 - The next step consists in applying Filippov's Lemma (cf. [H Theorem 8.2.10]). To do 
so, we define the map g : x A ^ as follows 

r 6i(X^o(t),ai) if X^o,iv(i) > 
g{t,a) := I b2{Xx^{t), 02) if ^xo,iv(^) < 
[ bn{XxAt),a) ifX,.„,jv(t) = 0, 

where a = {01,02, n)- We point out that we use here the general definition of 6-^, without assuming 
that bn{Xxo{t), a) -CN = 0. 

We claim that g is a Caratheodory map. Indeed, it is first clear that, for fixed t, the function 
a I—)- g{t, a) is continuous. Then, to check that g is measurable with respect to its first argument 
we fix a G A, an open set O C and evaluate 

g-\0) = {t>d:g{t,a)r\0^%] 

that we split into three components, the first one being 

g^\0) n {t : Xx,^N{t) < 0} = {t > : hi{Xx,{t),oi) G O} n {t : X^.o,7v(t) < 0}. 

Since the function t 1— > h\{XxQ{t),ai) is continuous, this set is the intersection of open sets, hence 
it is open and therefore measurable. The same argument works for the other components, namely 
{t : Xxo^wit) < 0} and {t : XxQ,N{t) = 0} which finishes the claim. 

The function 1 1— >• XxQ{t) is measurable and, for any t, the differential inclusion implies that 

Xx,{t)eg{t,A) , 

therefore, by Filippov's Lemma, there exists a measurable map a(-) = {01,02, G A such that 
(j2.3p is fulfilled. In particular, by the definition of g, we have for a.e. t G 

r hi{Xx,{t).ai{t)) if > 

Xx,{t) = l h2{Xx,{t),02{t)) iiXxMi)<^ (2-4) 
[ bu{Xx,{t),a{t)) iiXx,,N{t) = Q. 

Step 3 - The proof of {Hi) is an immediate consequence of Stampacchia's Theorem (cf. for 
example D. Gilbarg and N.S Trudinger [2T]) since, if y{t) := {XxQ{t))N, then y{t) = a.e. on the 
set {y{t) = 0}. □ 
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It is worth remarking that, in Theorem l2.1l a solution X^q (•) can be associated to several controls 
a(-); indeed in (j2.3p or (j2.4p the associated control is not necessarily unique. To set properly the 
control problem, without showing that (j2.4p has a solution for any a(-), we introduce the set Txq 
of admissible controlled trajectories starting from the initial datum xq 

%^ := {(X^^(-),a(-)) G Lip(M+;M^) x A such that (H^D is fulfilled and X^o(O) = xq} 

and we set 

£i := {t : € ni} , ^2 := : X,,{t) G J^s} , £h := {t ■ X,,{t) G H} . 

We finally define the set of regular controlled trajectories 

T^^^ := {{Xxq{-), a{-)) G Txo such that, for almost all t G Sy,, b'y{Xxa{t),a{t)) is regular}. 

Recall that, we call singular a dynamic b-}i{x, a) on T-L with a = (oi, 02, ^) when 5i(x, ai) • ctv > 
and 62(x, 02) • cat < 0, while the non-singular (or regular) ones are those for which the Oj) • e^v 
have the opposite (may be non strict) signs. 

The cost functional. Our aim is to minimize an infinite horizon cost functional such that we 
respectively pay li if the trajectory is in rij, i = 1, 2 and /-^ if it is on 7^. 

More precisely, the cost associated to {X3;o{-),a) G Txq is 

r+00 

J{xo;{Xx„a)) := / e{Xx,{t),a)e-^'dt 
Jo 

where the Lagrangian is given by 

e{Xx,{t),a) := h{Xx,{t),ai{t))l£,{t) + l2iXx,it),a2it))ls,{t) + lniXxoit),a{t))l£^{t) . 

The value functions. For each initial data xq, we define the following two value functions 

U-(xo):= inf J(xo;(X,„a)) (2.5) 

U+(xo):= inf J(xo;(X,„,a)) (2.6) 
(X.O'«)G7;o 

The first key result is the Dynamic Programming Principle. 

Theorem 2.2. Assume [HO], [HI] and [H2]. Let U~,U~^ be the value functions defined in ()2.5p 
and (j2.6p . respectively. For each initial data xq, and each time r > 0, we have 

U-(xo)= inf { r e{Xx,{t),a)e~'''dt + e-^^\J-{Xxo{T))] (2.7) 
U+(rEo)= inf \ r i{Xx,it),a)e~^'dt + e-^-U+iXx,{T))\ (2.8) 
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Proof. The proof is standard, so we skip it. 



□ 



Because of our assumption [H2] on bi, 62, it follows that -6(0, 6) C }3{x) for any x G M^. Hence 
the system is controllable, which means, roughly speaking, that the set of admissible controls is 
rich enough to avoid " forbidden directions" in any point of . 

The most important consequence of this is that both value functions U~ and U"*" are Lipschitz 
continuous. 

Theorem 2.3. Assume [HO], [HI] and [H2]. Then, the value functions U" and U"*" are hounded, 
Lipschitz continuous functions from into M. 



Proof. Since the proof is the same for U~ and U^, we denote by U a function which can be either 
U~ or U"'". We first notice that, if M is a large enough constant such that H^iHoo, ||'2||oo < M 
(recall that /i,Z2 are bounded), we have 

|U(z)| < ^ foranyzGM^, 
A 

therefore U is bounded. 

Next let X, y € and set K := We are going to prove that 

U(x) <\]{y) + K\x-y\ . 

Of course, if this inequality is true for any x,y G M^, it implies the Lipschitz continuity of U. 
To prove it, we assume that x ^ y (otherwise the inequality is obvious) and we set e 



\v-^\ ' 



By [H2], since B{0,5) C B{z) for any z G M^, it follows that 6e G B{z) for any z G and the 
trajectory 

Xxit) := X + Se ■ t , 

is a solution of the differential inclusion with Xx{0) = x and Xx{t) = y with r = . 
By the Dynamic Programming Principle 

U(x) < [ i{Xx{t),a)e-^^dt + e-^^lJ{y) , 
Jo 

and estimating the cost i{^Xx{t),a^ by M, we obtain 

V{x) - U(y) < Mt + (1 - e-^OI|U||oo • 

Finally, 



U(x) - U(y) < Mr + Ar||U||oo = - y\ 





and the proof is complete. 



□ 
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The Hamilton-Jacobi-Bellman Equation. In order to describe what is happening on the 
hyperplane we shah introduce two "tangential Hamiltonians" defined on 7^, namely HttH^^ : 
H xW X M^~^ — )• M. We introduce some notations to be clear on how they are defined: the points 
of H will be identified indifferently by x' G R-'^"^ or by a; = {x',0) S M^. Now, for the gradient 
variable we use the decomposition p = {p-HiPn) and, when dealing with a function u, we shall also 
use the notation D-^u for the (N — 1) first components of the gradient, i.e., 

du du / du \ 

Note that, for the sake of consistency of notations, we also denote by D-^u the gradient of a function 
u which is only defined on R^~^. Then, for any {x^u^py) G 7^ x M x R^~^ we set 

Ht{x,u,P'h) ■■= sup {-bn{x,a) ■ {pn,0) + Xu-ln{x,a)} (2.9) 

Aoi'x) 

where Aq{x) := {a = {ai,a2, fJ.) G A : by,{x, a) ■ cn = O} and 

HT^{x,u,pn) ■■= sup { -bn{x,a) ■ {pn,0) + Xu-ln{x,a)} (2.10) 



where A'^q^{x) := {a = (ai, a2, /^) G Aq{x) ; bi{x, ai) ■ < and b2{x, 02) • cat > O}. 

rTC 



The definition of viscosity sub and super-solutions for H-r an H^r^^ have to be understood on 



H, as follows: 

Definition 2.4. A bounded use funetion u : H —?■ M. is a viseosity subsolution of 

Ht{x, u, D-}{u) = on v. 
if for any (p G C^(M^~^) and any maximum point x'q of x' i— )■ u{x') — 4){x'), one has 

HT{xo,4>{x'Q),Dn4>{xo)) < , 

with xq = {x'q, 0). 

A similar definition holds for H^^, for supersolutions and solutions. Of course, if u is defined 
in a bigger set containing H (typically R^), we have to use uj-^ in this definition, a notation that 
we shall sometimes omit when not necessary. 

We first prove that both the value functions U~ and U'^ are viscosity solutions of the Hamilton- 
Jacobi-Bellman problem (|l.ip - (jl.3p - (|1.4p . while they fulfill different inequalities on the hyperplane 

n. 

Theorem 2.5. Assume [HO], [HI] and [H2]. The value functions and U"^ are both viscosity 
solutions of the Hamilton-Jacobi-Bellman problem 

Hi(x,u,Du) = in ill 

H2{x,u,Du) = in 0.2 (2 11) 

min \^Hi{x,u, Du), H2{x,u, Du)} < onH 
max {Hi{x,u, Du), H2{x,u, Du)} > onH. 
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Moreover, i— )■ U (x',0) verifies 

Ht{x,u,Dhu) <0 onn, (2.12) 

while x' U"'"(x',0) satisfies 

H'r^^{x,u,Dnu) <0 onn. (2.13) 

Remark 2.6. Once it is proved that JJ'^ is a viscosity solution of (ll.ip - (11.3p - (11.4p . then (j2.13p 
follows directly from Theorem \3.1l which concerns all suhsolutions of (jl.ip - (jl.3p - ()1.4p . However 
we give below a direct proof for . 

Proof. We start by proving that U~ and U"*" are both viscosity supersolutions of ()2.1ip . Let 
U = U"*" or U~. We consider (j) G C^(M^) and xq € such that U — (j) has a local minimum at 
xq, that is, for some r > we have 

U(xo) — (/)(a;o) < U(x) — 4>{x) \i \x — xq\ < r . 

We assume that this min is zero for simphcity, i.e. U(2;o) = (pixo). 

If xq G r^i (or ^2)) we can always find a time r small enough so that \Xx^{t) — xq\ < r and 
Xxgit) G Jli (or U2), for < t < r. Therefore the proof in this case is classical and we do not detail 
it. (See m, [5].) 

Now assume that xq ^ Ti and r is small enough so that \XxQ{t) — xq\ < r. By the Dynamic 
Programming Principle we have 

U(xo)= inf \ r £{X,,{t),a{t))e-^'dt + e-^^V{X^,{T))] , (2.14) 

where the inf is taken over 7^^ or Txq^ according to whether U = or U+. Thus 

^{xo)> inf \ r i{X,,it),a{t))e-'''dt + e-^^(^iXx,{T))] . (2.15) 

(^xo,a) I Jo J 

We use the expansion 

e-^^(/>(X,„(r)) =e-^V(xo) + {bi{Xx,{s),ai{s)) ■ Dct){Xx,{s))l£,{s) 
+ b2{X,,{s), a2{s)) ■ D(t>{Xx,{s))l£,{s) 

(2.16) 

+ bn{Xxois),a{s)) ■ D4>{X.,,{s))le^{s) 
-A(/.(X,„(s))}e-^^ ds. 

Using that l^j + l^-j + l^^ = 1 for the (— A0)-term, we rewrite (12.15P with three contributions 
(for simplicity, we drop the 's '-dependence in the integrands and use the inversion sup/inf) 

0< sup / \( -h{Xx„ai)-bi{X.x„ai)-D^{Xx,) + X(l){X^,))l£,{s) 



+ y - ^2(^x0, "2) - &2(^xo,a2) ■ ^<^(^xo) + A(/'(^xo)jl&(s) 

+ ( - IniXx^.a) - bniX^o^a) • D^iX,,) + A(/>(X,j)l^„(s) I e"^^ . 
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Since the Hamiltonians are defined as supremum of the various quantities that appear in the 
integrand, we deduce that necessarily 

0< sup [ \Hi(x,,{s),<PiX,,{s)),D<p{X,,is))]le,{s) 

(X,o,a)Jo [V / 

+ H2 (x,, is), <p{X,, is)),D<PiX,, is))] le, (s) 



+ H^(^X,,{s),^{X,,{s)),DnHX.o{s))ye^{s)^ e"^^ ds , 

where = Ht if U = U" and H:^ = H'r^^ if U = U+. Next, we use that Hi,H2,H^ < 
max{Hi, H2) together with l^-^ + l^^ + 1^^ = 1 so that we arrive at 

0< sup - I maxiHi,H2){X,,{s),c^{X,,is)),DcPiX,,is)))e-^'ds. (2.17) 

(X^Q,a) Jo 

Because of the regularity of (p and the continuity of the Hamiltonians we have that 

m&xiHi,H2){X^,{s),<PiX,,{s)),D<P{X,,is))) = max(i?i, i^a) (xq, (/-(xq), D(/'(xo)) + o(l) 

where o(l) denotes a quantity which tends to as s —> 0, uniformly with respect to the control. 
Therefore the sup in (j2.17p can be wiped out and sending r — > 0, we obtain 

max{Hi,H2){xo,(l){xo),D(j){xo)) >0 

which means in the viscosity sense that the supersolution condition is verified on T-L. 

Now we prove the subsolutions inequalities. We consider cf) € C^(M^) and xq G such that 
JJ — (j) has a local maximum at xq, that is, for some r > we have 

\J{xq) — (/)(xo) > U(x) — (/>(x) if |x — xol < r . 

Again, we assume that this max is zero for simplicity. 

Here also, if xq G 0,i (or ^^2)) we can always find a time r small enough so that \Xxq (t) — xq] < r 
and Xxoit) G ^^i (or Q2) for < t < r. In this case the proof is classical (See [4], [5]). So, assume 
that Xq (z H, T is small enough so that {Xx^it) — xq\ < r for t < t. By the Dynamic Programming 
Principle we have 

Vixo)= inf \ r i{Xx,{t),a{t))e-^'dt + e-^^V{Xx,{T))] , (2.18) 

thus 

(t>{xo)< inf \ r e{X,,{t),a{t))e-^'dt + e-^^(t>{X,,{T))] . (2.19) 

We distinguish now 5 sub-cases. Notice that since the inf is taken on Txq^ for U+, the third 
possibility below does not occur in this case. 
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Case 1 - Let ai,a2 £ Ahe any constant control such that 61 (xq, ai)-eN > and b2{xQ, a2)-e]\[ > 0. 
Then there exists a time r such that the controlled trajectory (Xxo,a) lives in r^i, for all s €]0,t]. 
Therefore, by the inequahty (|2.19|) . the expansion (|2.16|) and classical arguments (dividing by r 
and letting r ^ 0), we obtain 

- bi{xo, ai) ■ D(l){xo) - h{xo, ai) + Xcpixo) < . (2.20) 



Case 2 ~ Let ai,a2 G ^4 be any constant control such that 61 (xq, ai)-e]\f < and b2{xo, 02) -e^ < 0. 
By the same argument as in case 1) we obtain 

- 62(xo, 02) • D^{xo) - hixQ, 02) + A0(xo) < . (2.21) 



Case 3 - Let ai, 02 S ^4 be any constant control such that 6i(xo, ai) -ctv > and b2{xQ, 02) -e^ < 
(we can allow here also the case of one of the two to be zero). There exists then a trajectory 
{XxQ,a) G Txo such that Xxq{s) € H for a small time r. Indeed, if y € is close to xq and 
fj, = fj,{y) is defined as follows 

fi{y) ■= -62(^,02) • eN 

(bi{y,ai) - b2iy,a2)) ■ cn' 

we consider the solution of x{s) = fj.{x{s))bi{x{s),ai) + (1 — fi{x{s)))b2{x{s),a2), x{0) = xq. By 
the regularity of bi and 62 (and thus of n{y)) the Cauchy-Lipschitz Theorem applies and it is easy 
to check that, by the definition of /.t, this trajectory lives in Ti in the interval [0, r], for r small 
enough. Moreover, by the signs of 6i(xo,ai) • cn and b2{xo,a2) ■ cat, we have < /i(x(s)) < 1 and 
therefore, x(-) is a controlled trajectory associated to a = (ai, 02, /i(x(-))) on [0,r]. By Inequality 
(j2.19p . the expansion (|2.16p and classical arguments, we obtain 

- bnixo, a) ■ D(t){xo) - Inixo, a) + X^{xo) < . (2.22) 



Case 4 - Let ai,a2 G ^ be any constant controls such that 61 (xq, ai)-e]\f < and 62(2:0) 02) -sn > 
(here also we can allow the case of one of the two to be zero). By the same argument as in case 3) 
we obtain that (j2.22p holds. 



Case 5 - Let ai, 02 € A be any constant control such that 61 (xq, ai) -6^ = and b2{xo, 02) ■ ejy = 
0. By the controllability assumption [H2], there exist aj~,a^ such that 6i(xo,aj~) = —Scn 
and 62(3:0)02') ~ {6 > given by [112]). For < 77, 77' < 1, by the convexity assump- 
tion in [H2], we can find and such that 6i(xo,a^) = r?6i(xo,aj~) + (1 — 7/)6i(xo,ai) and 

62(xO)a2 ) ~ ^62(2:0)02) + (1 ~ ?/)62(xO) 02)) respectively. Therefore 6i(xO)a^) • cn = —7]5 < 
and 62(xo, Og) • = rj'S > 0, and, by arguing as in case 4), we obtain again that ()2.22p holds with 
a = (a^,Q2 ,fl{r],r]')) where 

fl{r], r]') := -62(^0, e^y ^ rj' ^ 

(6i(xo,a^) - 62(xo,a^ )) • CAT ?? + ?/ 

for all < 77, 77' < 1. By construction, 6i(xo,ai) — > 6i(xO)ai) and 62(xO)Q!2 ) — ^ 62(xO)Q!2) as 
77, 77' — >• and, on the other hand, given some < ^ < 1, we can let 77,77' tend to in such a way 
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that fj,(rj, 7]') —7- /U. Then, ()2.22p holds for any < /i < 1 by letting 77, r]' — > in a suitable way, 
recalling also the continuity of bi and 62- 

By remarking that cases 3, 4 and 5 imply that either —bi{xo, ai)-D(p{xo)—li{xQ, ai)+\(p{xo) < 
or —b2{xo,a2) ■ D(P{xq) — l2ixo,a2) + A(/>(xo) < 0, and that we classified all the possible constant 
controls we can conclude that 

mm{Hi,H2){xo,(l)ixo),D(l){xo)) < 
and the proof of (j2.1ip is complete. 

Let us now prove that U" verifies ([21^ . We consider then (/> G C^(E^~^) and xq = {xq,0) £ U 
such that x' ^ U~(x',0) — (j){x') has a local maximum at x'q. We assume that this max is zero 
for simplicity and we extend the test function as follows: (j){x',XN) = (t>{x') which is a C"^(M^) 
function, independent of the A^-th variable. Notice that D(p{x) = (Z)^(/>(x'), 0). 

If (X^.o, a) is a controlled trajectory such that Xxgit) € 7^ for t G [0, f] for some f > 0, we have 
for < r < f, by the Dynamic Programming Principle 

V-{xo) < r i{X,,{t),a{t))e-^'dt + e-^-V-{X,,{T)), 
Jo 

which implies 

4>{xo)< [ £{X,,{t),a{t))e-^'dt + e-^^4>{Xxo{r)) . 
Jo 

The proof follows the same arguments as before in the proof of U~ being a subsolution of (j2.1ip 
for the cases 3,4 and 5 (for which we have indeed XxQ^t) G T-L for t G [0, f] for some f > 0). In other 
words, since we are considering only the controls in ^0(2^0)) we do not have controls fulfilling cases 
1 and 2. Therefore, all the possible controls in Aq{x()) are considered for which we obtain (j2.22p . 
Thus, 

sup { - bn{xo, a) ■ D(f){xo) - /^(xq, a) + X(f){xo)] < , 

a£Ao{xo) 

that we interpret as follows: 

HT{xo,(p{xo),Dn4>{xo)) = sup { - bn{xo,a) ■ {D-h(I){x'q),0) - ln{xo,a) + X(I){x'q)} <0, 

aeAo{xo) 

hence ()2.12p holds. In order to prove that U+ verifies ()2.13p we argue exactly as before remarking 
that we do not have to consider cases 1, 2 and 3. □ 

In what follows, we are going to consider control problems set in either r^i or ^1,2 (or their 
closure). For the sake of clarity we use the following notation. If xq G Oj, {i = 1,2) and ai{-) G 
L°°(M+;^j), we will denote by Y^^{-) the solution of the following ode 

Yi^is) = h{Y:^{s),a,{s)) , Y:^{0) = xo. (2.23) 
Our next result is a (little bit unusual) supersolution property which is satisfied by U"*" on H. 
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Theorem 2.7. Assume [HO], [HI] and [H2]. Let (f) € C^(]R^ ^) and suppose that x'q is a minimum 
point of x' I— > U~'^(a;',0) — (f){x'). Then we have either 

A) There exist i] > 0, i = 1 or 2 and a control ai{-) such that, if xq := (xq,0), Y^^{s) € Qi for all 
s €]0, T]] and 

U+(xo) > rk{Yi:^{t),a,{t))e''''dt + V~^{Y^^{v))e-^^ (2.24) 



or 

'B) it holds 

H'T'^xo,\J+{xo),Dn^{x'o))>0. (2.25) 

We skip the proof of this result to reduce the length of our paper since it is similar to the proof 
of Theorem [ 



3 Properties of viscosity sub and supersolutions 

In this section we describe the properties fulfilled by the sub and supersolutions of system (jl.ip - 
(jl.3p - (ll.4p . We are going to consider only bounded sub and supersolutions, a natural class according 
to Section [2j Because of [H2] , the subsolutions are automatically Lipschitz continuous since the 
Hamiltonians are coercive but, a priori, the supersolutions may be only lower semicontinuous. 

We first prove that any subsolution of (ll.ip - ()1.3p - (ll.4p is a viscosity subsolution of H^^ = on 
the hyperplane H. 

Theorem 3.1. Assume [HO], [HI] and [H2]. If u : ^ R is a bounded, Lipschitz continuous 
subsolution of (|l.ip - (|1.3p - ()1.4p . then u is a subsolution of \2.13\) . 

Proof. Let </>(•) be a C^-function on R-^"^ and x'q a maximum point of x' ^ u{x' ,Q) — (f){x'), our 
aim is then to prove that, for any a € A^q^{xo) where xq = (xg, 0), we have 

- bnixo, a) ■ {DnHx'o), 0) - Inixo, a) + \u{xo) < 0. (3.1) 

We first remark that it is sufficient to prove this inequality for the elements a of A^q^^xq) such that 

6i(xo,ai) • Cat < 0, 62(3^0, "2) • cat > and {fibi{xo,ai) + (1 - At)&2(a;o, "2)) • cat = 0. (3.2) 

Indeed, the case of non-strict inequalities can be recovered thanks to assumptions [II0]-[II2], with 
the same argument as in the proof of Theorem 12.51 (|2.13p -CASE 5. 

We fix now any triple {01,02, fJ-) such that ()3.2|) is fulfilled and, as usual, we define the function 
(f){x' ,xn) ■= (j){x') for which D(]){xq) = (L)-^0(x'), 0). 

For < e ^ 1, we consider the function 

x^ 

X I—)- u(x) — (f)(x) — r]XN ^ ~ 1^^ ~ ^oP := u{x) — i[e{x) (3-3) 

where the constant r/ € M is chosen as follows: we consider the solution f/ € M of 
-61 (xo, oi) ■ {D(t>{xo) + r/eAf) - /i(xo, ai) + Au(xo) = . 
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Such a solution exists because of Property (j3.2p on 61 (xq, ai) and we choose r] > r]. Therefore 
-6i(xo, ai) • {D^{xo) + r/CTv) - /i(a;o, ai) + An(a;o) > . 

By standard arguments, the function u — ips has a local maximum Xe in and ^ xq as 
e — 7- 0. We want first to show that for e > small enough (the other parameters being fixed for the 
moment), Xe necessarily belongs to due to the penalization. So, assume on the contrary that 

G Oi. Since ti is a subsolution of ()l.l|) - (|1.3p - ()1.4p . we have 

HiiXe,uiXe),D^e{Xe)) < , 

which implies 

- bi{Xe,ai) ■ DipeiXe) - /i(Xe,Qi) + Xu{Xe) < . (3.4) 

But DiIj^Ix^) = Dcf>{xs) + rjCN + 2 ^ ^1^ + 2(xs — xq) and therefore 

-6i(xe,ai) ■ Di/jsixe) = -bi{xe,ai) ■ {D4){xo) +r]eN) - 2 ^^"]^ bi{xe,ai) • bat + 0^(1), 

where 0^(1) is a quantity going to zero as e — )• 0, the other parameters being fixed. But, because 
again of Property (13. 2p and the fact that x^ S f^i which implies (xe)Ar > 0, we have 

-2 5— fti(Xe,ai) • BAT > . 

Finally, recalling the continuity of 61, /i, and n, we deduce that 

-6i(Xe,ai) • DlpeiXe) - /l(Xe,Qi) + A'u(Xe) > 

-61 (xo, ai) • {D(f>{xo) + 7?eAr) - /i(xo, ai) + An(xo) + Oe(l) . 
Our choice of t] implies that Inequality ()3.4p cannot hold for e small enough, and therefore x^ S Jl2- 
In the same way, if x^ £ T-L, we have 

mm{Hi{Xe,u{Xs),Dlpe{Xe)),H2{Xs,u{Xs),Dlpe{Xe))) <0 (3.5) 
but the above proof shows that, for e small enough, 

HiiXe,u{Xe),DTpe{Xe)) > , 

and therefore 

H2{Xe,u{Xe), DtpeiXs)) < . 

In particular, this implies 

- b2ixs,a2) ■ Dipsixe) - l2ix£,a2) + XipeiXe) < . (3.6) 
Now Xe G ^2 which implies {xe)^ < and invoking again Property (j3.2p . we have 

-2 — —b2(x£,ax) • CAT > , 
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since b2{xi;,ai) ■ ejy >0 for e small enough. This yields 

-62(3:0, "2) • (DH^o) + r/CAr) - hixo, ai) + Xu{xo) + Oe(l) < . 

In this inequality, we first let e tend to and then r] tend to ij. 
In order to conclude, we use the value of f], namely 

- ^ -61 (xq, ai) ■ D(f>{xo) - Ii{xq, ai) + A-u(xo) 
bi{xo,ai)-eN 

and an easy computation on the inequality 

-62(2:0, 02) ■ {D4>{xo) + fjcN) - hixo, ai) + Xu{xo) < , 
provides the desired inequality. □ 



Now we prove two properties verified by sub and supersolutions in the domains that will be 
important to obtain the uniqueness results. 

Lemma 3.2. Assume [HO], [HI] and [H2]. Let v : M be a Isc supersolution of (frT]) - (fr3l) - 

(jl.4p . and u : — ?> M 6e a Lipschitz continuous subsolution of (|l.ip - (|1.3p - (jl.4p . Then, if xq G Oj, 
we have for all t >0 



v(xn) > inf 



k{Y:^{s),a,{s))e-''ds + viY^^it A e,))e 



(3.7) 



and 



u(xo) < inf 



kiYUs), a.is))e-^^ds + u^YUt A 0,))e-^(*^^») 



(3.8) 



where Y^^ is the solution of the ode (|2.23p and the infima are taken on all stopping time 9i such 
that Y^g(9i) S H and ti < 9i < fi where ti is the exit time of the trajectory Y^^ from and fi is 
the one from Jlj . 



Proof. For x = u or v, we consider the exit time-Dirichlet problems 

wt + Hi{x, w, Dw) =0 in X (0, +00) 
< w{x,Q)=x{x) on Hi (3.9) 

w{x,t) = x{x) on dQi X (0, +00) . 

The proofs of (j3.7p and (j3.8p are slightly different. Property ()3.7p directly follows from the results 
of Blanc [9l [10] since is a supersolution of ()3.9p (with x = while the right hand-side is the 
formula for the minimal supersolution (and solution) of this problem. It is worth pointing out that 
(i) we do not need relaxed controls because of Assumption [112] since, in our case, relaxed controls 
coincide with usual L°° controls and (ii) the results in [HI [TO] are obtained in bounded domains but 
they can easily be extended to unbounded domains. 
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For ()3.8p . the right-hand side of the inequahty is a supersolution of (|3.9p (with x = u) while u 
is a subsohition. The comparison in fij of these sub and supersolution follows from the result in 
Barles and Perthame [7] (see also [S]) because u is (Lipschitz) continuous. The continuity of n is a 
key point in the comparison property. □ 

We prove now a property fulfilled by a supersolution v of (ll.ip - ()1.3p - (ll.4p which highlights the 
alternative between, roughly speaking: (i) there exist an optimal strategy consisting in entering 
one of the sets i}i/i}2', (ii) the optimal strategies consist in staying on 7i for a while. 

Theorem 3.3. Assume [HO], [HI J and [H2j. Let v. R be a Isc supersolution of (frT]) -(fL3ll- 

()1.4|) . Let 4> G C^(M^~^) and x'q be a minimum point of x' i— v{x',0) — 4>{x'). Then, either 

A) there exist r] > 0, i = 1 or 2 and a sequence G fij — > xq = (2:0,0) such that v{xk) v{xq) 
and, for each k, there exists a control a^{-) such that the corresponding trajectory Y^^{s) € for 
all s G [0, 7]] and 

v{xk) > rk{Yi^{t),a'[{t))e-''dt + v{Yl{rj))e-^^ (3.10) 
Jo 

or 

B) it holds 

HT{xo,v{xo),Dn(t>{x'o))>0. (3.11) 

Proof. We are going to prove that, if A) does not hold, then we have B). To do so, we first remark 
that, changing cl){x') in (f){x') — \x' — Xgp, we can assume that Xq is a strict local minimum point 
of v{x',0) — ([>{x'). We then define the function 4>{x',xn) '■= ([>{x') in order to have D4>{xq) = 
(l)«</)(a;^),0). 

For e > which is devoted to tend to 0, we consider the function 

vix , xn) - (t>ix' ,xn) - SxN + 

where (5 € M will be chosen below. By standard arguments, this function achieves its minimum 
near and, any sequence of such minimum points x^ converges to xq = {x'q, 0). 
l'^* case. Let us first suppose that, for all 5 G M, the minimum is attained at a point of 
the hyperplane H. Thus, because x'q is a strict local minimum point of v{x' ,Q) — 4>{x'), then 
x^ = (3:0,0) = xo and, since v \s a supersolution of (|2.1ip . we have 

Lp{5) \=\i\&-K{Hi{xQ,v{xQ),Du(t){x'Q) + 5eM),H2{xQ,v{xQ),Du(p{x'Q) + 5eM)} >0 (3.12) 

for any (5 G M. Notice that, for the sake of simplicity of notations, we have written D'^(j){x'Q) + 5eN 
instead of {D'j-i4){x'q) , 5) . 

The function ip{-) defined in such a way is convex and coercive (since Hi, H2 are convex and 
coercive) and, if 5 is a minimum point of 93, we have G dip{6). 

By a classical result (see the book of Rockafellar |23j). since 99 is expressed in terms of supremum 
of quantities like 

-bi{xo, ai) ■ {Du4>{x'q) + Scn) + \v{xq) - Ii{xq, ai) , 
-b-iixQ, a2) ■ {Du4>{x'q) + Scn) + \v{xq) - hixQ, a2) , 



17 



then any element of the subdifferential of ^p, is a convex combination of the gradients of such 
functions (namely — 6i(xo,ai) • e^v and —62(3^0102) • e^) for the ai,a2 such that 

if{5) = -bi{xo, ai) • (DncpixQ) + Jctv) + Xv{xo) - li{xo, ai) , 

ip{5) = -b2{xo, 02) ■ {Du4>{x'q) + Scn) + \v{xq) - hixQ, 02) , 

i.e. for the ai, 02 for which the maximum is achieved. 

Taking [H2] into account and looking at the meaning of these properties at the point 5, we see 
that it can be reduced to: there exists a ;U € [0, 1] such that 

= /x(6i(xo,ai) • cat) + (1 - fJ.)ib2{xo,a2) ■ e^) , and 

< ip{5) = fi{- biixo, ai) ■ {D'^4){x'q) + Scn) + \v{xq) - li{xo, ai)] 

+ (1 - fi){ - 62(2:0, "2) • (DncpixQ) + 6eN) + >^v{xo) - l2ixo,a2)}. 

Inequality easily follows. 

2"^^ case. As a consequence of the arguments which are used in Case 1, we see that, if tp{S) > 
where, as above, is a global minimum point of if, we are done. Hence we may assume ip{5) < 0. 

We consider the function 



X 



ll^six) ■■= 4>(,X) + 6XN n- 



and denote by x^ a minimum point of v — ips- 

Since ip{5) < 0, Xg cannot be on Ti. Therefore we can apply Lemma 13.21 which gives, for any 



t > 



v(xr) > inf 



(3.13) 



where we denote by K^^(-) the solution of the ode (j2.23p starting from x^ at time 0. 

Because of [H2], the infimum in (I3.13p . say for t = 1, is attained for some af(-) and 6f > 0, 
namely 



v{Xs) > 



lA6»f 



(3.14) 



kiVlis), a\{s))e-^^ds + v{Yi^{l A ))e-^(^^^»^) 

Moreover, recalling that we are assuming that A) does not hold, we have that 0? — >■ as e — )> 0. 
But using that is a local minimum point of v{x) — ipe{x) we deduce that, for e small enough 



> 



k{Yl{s),a,{s))e-^''ds + v{YU9t)){e 



-A0f 



i) + UYiiei))-u^,). 



(3.15) 



Next we remark that, since by definition (Y^^{91))n = 0, we can drop the quadratic term in tp^ 
Indeed 



< . 



Therefore, inequality p.lSp becomes 



> 



k{Yl{s),al{s))e~^^ds + v{Yi^M))i^~^'^' " 1)+ 

Dn<p{Yl{s)) + 5.eN 



bi{Yi{s),aUs))ds, (3.16) 
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thus 
< 



h(YUs),c4{s))e 



-Xs 



DHHyiis))+S-eN 
viY:{9!)){e-''l -1)< 



b,iYUs),alis))\ds 



< I sup 

/o a^eA 



k(Yi(s),o4)e 



-Xs 



Dn4>{Yl{s)) + ~8-eN 



h{Yl{s),al)^ds 
-v{Yl{9t)){e-^'t -I). (3.17) 
If we divide now by Of and let e tend to 0, we obtain by usual arguments 

< sup {-/i(xo, ai) - {Du(t>{xQ) + 5 • cat) • hi{xQ, Oj)} + \v{xq) 
= Hi{xQ,v{xQ), Du<P{x'q) + 6 ■ cat), 



which is a contradiction, so that the proof is complete. 



□ 



4 A uniqueness result. 

Theorem 4.1. Assume [HO], [HI] and [H2]. Let u he a hounded, Lipschitz continuous suhsolution 
of prT]) - (fL3]) - (fLll) - (fT3]) and v he a hounded, Isc supersolution of ([□])- ([L3])-([LlD. Then u < v in 



Proof. In order to justify our strategy of proof, we point out that the usual "doubling of variables" 
method, which is very classical in viscosity solutions' theory, cannot work here since if we look at 
a maximum of u{x) — v{y) — • • • , then x and y can be in two different part of the domain (either 
or r22) and we would face two completely different and therefore useless inequalities. Therefore 
we have to look at a maximum of u{x) — v{x) and to do so, we have to (i) regularize u to make 
it at least in the xi, - ■ ■ ,xn-i variables and (ii) manage to turn around the difficulty of the 
non-compact domain. 

The regularization of u relies on (almost) standard arguments. We use a sequence of mollifiers 
{pe)e defined on as follows 



where 



p G C7°°(M^"i), I p{y)dy = 1, and suppM = B^n-^{G, 1) 



Next we consider the function Ug. defined in by 



Ue{x) :-- 



u{x' — e,xj\[)Pe{&)de 



A key result is the 
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Lemma 4.2. There exists a function m : (0, +00) (0, +00) with m(0+) = such that the 
function — m(e) is a viscosity suhsolution of p.ip - p.3p - ()1.4p - ()1.5p . 

We skip the proof of this result which rehes on standard arguments: see the book of P.L Lions 
|22j or Barles & Jakobsen [6] . It is worth pointing out that it is completely standard for (jl.5p which 
is an equation set in M^-\ a little bit less classical for (jl.ip . We use in a crucial way the fact that 
u is Lipschitz continuous, as a consequence of the controlability assumption [H2] (which implies 
that Hi, H2, Ht are coercive Hamiltonians) . Of course, m(e) is a quantity which controls the error 
terms through the An-term. 

Next we have the 

Lemma 4.3. For M large enough, the function ip^x) := — jxp — M is a viscosity suhsolution of 

Again the proof is easy since the assumptions [H0]-[H2] implies that for H = Hi, H2, Ht we 
have 

H{x,t,p) < Ci\p\ + Xt + C2 . 

Therefore we just have to estimate Ci|2x| + A(— |a;p — M) + C2 and the conclusion follows easily 
by using the Cauchy-Schwarz inequality for the first term. 

Using these lemmas, the proof of the result is easy: for < /x < 1, close to 1, we set ti^^^ := 
fi{ue — m{e)) + (1 — n)ip. Notice that, by the convexity properties of Hi, H2, Ht, tte,/^ is still a 
suhsolution of (ll.ip - (ll.5p . Moreover Ue,fi{x) —00 as \x\ ^ +00. 

Therefore we may consider M^^^ := maxjgjv(ne^^(3;) — v{x)) which is achieved at some point 
X G and we argue by contradiction assuming that M^^^j > 0. 

We first remark that, necessarily, x (z Ti. Otherwise, we introduce the function u^^^{x) — 
v{x) — \x — which has a strict maximum at x and we double the variables, i.e. we consider, for 
< /3 < 1, 

I |2 

{x, y) ^ Ue,i,{x) - v{y) - ^ \x - x\^ . 

Applying readily the classical arguments and remarking that the maximum points of this function 
converge to {x,x), we would be led to the conclusion that ^^^^(x) < v{x) and therefore M^^^ < 0. 
A contradiction. 

Since x ^ T-L, we can turn to Theorem 13. 3i We point out that u^^^ is with respect to the 
xi, - ■ ■ , X7V-1 variables and therefore ti^^^ is both a test-function for the u-inequality and it satisfies 
the suhsolution inequality in the classical sense. Either we are in the B) case, Ht{x, v{x), Dy^Us^^ix)) > 
0, and we conclude with a classical comparison result that < v{x) since 

HT{x,Ue^^{x), DuUe^^i{x)) < . 

Or we are in the case A). Therefore, for any k, we have at 

v{xk) > r h{Y:^{t),aHt))e-''dt + v{Y:^{v))e-^'^ 
Jo 
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and 

UeA^k) < [\^{Yi^{t),a'l{t))e-^'dt + u,,,{Yi^{v))e-^'^ , 
Jo 

where the last inequahty is a consequence of Lemma 13. 2i Subtracting these inequahties gives 



and letting k tends to +00 yields 

a contradiction which proves that Mg^^ < 0. 

This means that Us,^ < f in and we conclude by letting first tend to 1 and then e tend 
to 0. □ 

Corollary 4.4. Assume [HO], [HI] and [H2]. 

(i) U~ is the minimal supersolution and solution of il.l\) - [T73\) - ^1.4\ )- 
(a) is the maximal suhsolution and solution of 

(Hi) There exists a unique solution of hl.l\) - [L3) - ^.4\ )- ll-^) and for this problem, any suhsolution 
is below any supersolution (Strong Comparison Result). 

This corollary can be read in the following way: unfortunately, we do not have a Strong Com- 
parison Result for (jl.ip - (|1.3p - (jl.4p but we can identify the minimal supersolution (and solution) 
and the maximal suhsolution (and solution) . But if we add the suhsolution condition (ll.5p , then we 
recover the full Strong Comparison Result. In Section O explicit 1-D examples show that this result 
is optimal and we give also sufficient conditions in 1-D which ensure that that (ll.ip -( fL3]l - (]1.4p and 
([lT])-([r3])-([LlI)-([I3]) are equivalent. 

Proof. The proof is very short since we have just to apply Theorem 14.11 for most of the cases. 

The result (i) is an immediate consequence of the fact that U~, is a suhsolution of (jl.ip - (]1.3p - 
(jl.4p - (|1.5p , while, for (iii), the result is exactly Theorem 14. 1[ 

To prove (ii) we aim to use the same arguments as in the proof of Theorem 14 . 1 1 with Hamiltonian 
Ht being replaced by H^^. 

Let u be any bounded Lipschitz continuous suhsolution of ()l.ip - (jl.3p - (jl.4p . By the above regular- 
ization, we can suppose that it is a function on Let x be a maximum point of u — U"^ 

on M^, and observe that, as above, we can assume that x ^ H. We observe now that the function 
fulfills Theorem 12.71 at point x, mimimum of the function x' ^ U+(x', 0) - u{x' , 0). (Note that 
this is the analogue of Theorem 13. 3p . Therefore, we have case A) or case B). 
If we are in case B), it holds 

H'T'^x,U+{x),Dnu{x)) >0, 

therefore, the conclusion easily follows observing that by Theorem 13.11 u fulfills in the classical 
sense 

Hp{x,u{x),Dnu{x)) < 0. 

If we are in case A), there exist ry > 0, i = 1 or 2 and a control ai{-) such that, Y^(s) S flj for all 
s s]0, T]] and 

U+(x) > rk{Yi{t),a,{t))e-''dt + V+{Yi{rj))e-^^. 
Jo 
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Therefore the conclusion follows by applying (j3.8p in Lemma [3 .21 noticing that, since u is Lipschitz 
continuous, this property can be extended to points of VLi (and not only to points of Vti). 

□ 

Remark 4.5. The last part of the proof of Corollary \4.4\ clearly emphasizes the uniqueness problem 
with l[l.l\) - l[T73\) - l^l.4\ ): either, on Ti, we allow the singular strategies 

{fj,bi{x, ai) + (1 - fJ.)b2{x, 02)) • Cat = 0, 61 (x, qi) ■ cn > 0, 62(2;, ^2) • cat < , 

and, since they are not encoded in il.l\ )- [T73\ )- l^l.4\ ), we have to add lil.5\) to get the uniqueness. 
Or we do not allow them (or they are not optimal) and we obtain the uniqueness. In any case, the 
choice of the inequality Ht (to be imposed) or H!^^ (consequence of U.1\ }- [T73\ )- I^1.4^ ) makes the 
difference. 



5 The 1-D case 

In this section we go a little bit further by providing a complete classification of the value functions 
U~ and U+. Moreover, we derive explicit examples highlighting the different strategie that we call 
"state constraints" , "push-push" and "pull-pull" strategies, and the non-uniqueness phenomenon. 



5.1 Structure of solutions 

In order to describe the structure of solutions, we introduce the state constraint solutions Usci in 
0,1 and Usc2 in ^^2 which are defined, for i = 1,2 in the following way 



Usci(a;o) := inf 
.4sci 







+00 

k{Yi^X'),a,{s))e-^'ds 



where ^sci is the set of controls ai(-) for which Y^^{s) € for any s > 0. Note that by the 
classical results in [25], Usci are solutions of Hi = on J7j and Hi > on (see also jlj). 

We also denote by 

un{0) = \^ niin |/i/i(0, ai) + (1 - /_f)/2(0, 02)) , 

which is the solution of Ht{x,u, D-^^u) = on Ti = {0} in this particular case. Similarly, u^^(O) is 
the same quantity when the min is taken over the regular controls, A^q^{0). 

Theorem 5.1. Assume [HO], [HI] and [H2]. The following holds U-(0) = min {^-^(0), Usci(O), Usc2 
Therefore we have 

(i) if the min is given by u-^(O), then U~ is the unique solution of the Dirichlet problems 

{Hi{x,w, Dw) = in ^li \ H2{x,w, Dw) = in ft2 

and < 
w{0) = un{0) \w{0)=un{0) 
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(ii) if the min is given by Usci(O), then U = Usci i^ ^i, <ind in 0,2 it is the unique solution 
of the Dirichlet problem H2 = with boundary value U~(0) = Usci(O); 

(ii) if the min is given by Usc2(0), then = Usc2 in ^2, <ind in 0,i it is the unique solution 
of the Dirichlet problem Hi = 0, with boundary value U~(0) = Usc2(0). 

Similarly, U^(0) = min {li^^(O), Usci(O), Usc2(0)} and the value of the min identifies as 
above, among the three possibilities. 



Concerning uniqueness, a direct consequence is the following: 



Corollary 5.2. Assume [HO], [HI] and [H2] and one of the following condition holds: {i) u-^(O) = 
w5^^(0); {ii) n-^(O) > min |Usci(0), Usc2(0)} . Then U~ = U+.- there is a unique solution of the 
problem. 



Theorem 15.11 follows from the conjunction of several results. We begin with the following 
proposition concerning supersolutions. 

Proposition 5.3. Assume [HO], [HI] and [H2]. Let v be a bounded, Isc supersolution of 
([I3j-([73P. Then 

i;(0) >min{u^(0),Usci(0),Usc2(0)} =U-(0). 

Before proving this proposition, we go back to the solution introduced in Lemma 13.21 which is 
considered here for xq ^Q-i 



r rtf\9 



vixo) > W-{xQ,t) := inf 



' h{Yl{s),aiis))e-'^ ds + v{Ylit A 9i))e~'('^'^^ 



(5.1) 



where we recall that Y^^ is the solution of the ode (j2.23p with i = 1 and the infimum is taken on all 
stopping times 0i such that Y^^^{9i) = and ti < 61 < fi where ri is the exit time of the trajectory 
Y^^ from J7i and fi is the one from 

We again point out that we do not need relaxed controls in the expression of w- because of 
Assumption [H2] since in our case, relaxed controls coincide with usual L°° controls. Using this 
remark, we notice that, for any xq 7^ and t > 0, the above infimum for w- is achieved for some 
control ai(-) and some stopping time < ^1 < +00, namely 

w.{xo,t)= / h{Yl{.s),ai{s))e-''ds + v{Yl{tAei))e-'(''^'-K (5.2) 



A priori, ai(-) and 61 depend on xq and t but we drop most of the time this dependence for the 
sake of simplicity of notations. 

The following lemma holds 

Lemma 5.4. Either v(xo) > Usci(xo) or, fort large enough, the above defined control ai(-) and 
stopping time 9i can be chosen as being t-independent, 9i being finite. Moreover the associated 
trajectory Y^ (■) is decreasing. 
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Proof. We assume that v(xq) < Uscil^^o); the aim of the proof is to show that the second case is 
true. 

We first remark that, necessarily, this imphes that the stopping time 9i (which a priori depends 
on t at this stage of the proof) remains uniformly bounded as t ^ +00. Otherwise, at least 
up to some subsequence, we can pass to the limit in (j5.2p . using the compactness of controlled 
trajectorie^ and we get, by standard arguments, that 



limsup ^^-{xo^t) > inf 



h{Yl{s),ai{s))e-^' ds 



Usci(a:;o) > v{xo) 



which contradicts (15.11). 



Therefore we may assume that 9i remains bounded and take t large enough so that t A61 = 61. 
Then 



w-{xo,t) 



hiYlis),aiis))e-^-'ds + viYl{ei))e-^'^ 



inf 



h{Yl{s),ai{s))e-^'^ds + v{Yl{e,))e-^'- 



the last equality being a consequence of the optimality of the control ai (•) and the stopping time 9i. 
For this infinite horizon, exit time control problem, there exists an optimal control and a stopping 
time that we still denote by ai(-) and 9i (which are obviously independent of t). 

Moreover, pick any < s < ^1, by the Dynamic Programming Principle for W- 



W-{xo,t) = inf 

ai(-),6»i 



(t-s)A6»i 



< 



h{Y,\{s),a,{s))e~^'ds + w.{Yl{{t - s) A ^i),* - s-)e-^«*-^")^''i) 
h {Yl (s) , ai is))e-^'ds + {Y^ {s),t- s)e-^' . 



We deduce from this property that 



W-iY^oi^),t- s)e- 



h {Y^ is) , ai {s))e-^^ds + v{Yl {9i))e-''^ 



Therefore 'W-{Y^^{s),t — s) is independent of t as well and we drop this dependence by just writing 
w_{Y^^{s)) for < s < ^1. 

If s 1-^ Y^^{s) is not monotone on [0, there exists < si < S2 < 9i such that Y^^{si) = 
Y^^{s2)- By the above property, we have 

w.{Yl{si))e~>^'- = rh{Yl{s),ai{s))e~^'ds + w.{Yl{s2))e-''' . 



Using the fact that Y^^[si) = Y^^{s2) and the Dynamic Programming Principle which can be 
written as 



xo \ 
si 



w.{xo,t)= / h{Yl{s),ai{s))e-^'ds + w.{Yl{si))e-^'\ 



^We recall again that we do not need relaxed controls because of Assumption [H2]. 
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this means that, iterating the loop, from si to S2, we have an optimal control defined for all s > 0. 
More precisely we introduce the control 



ai(s) 



ai{s) if < s < S2 , 

ai{s - {S2 - Si)) if S2 < S < S2 + (S2 - Si) 



ai{s - k{s2 - si)) if S2 + k{s2 - si) < s < S2 + {k + l)(s2 - si) . 

Arguing by induction, it is easy to show that the associated trajectory Y^^ remains in J7i for all 
s > 0, that Yl{s2 + {k + 1){S2 - s,)) = Y^isi) = Yl{s2) and 

rS2+{k+l){s2~si) 

W-{xo,t) = / /i(yi„(s),di(s))e-^^ds + «;_(yi„(s2))e-^(^^+('=+i)(^^-^^)) . 

Jo 

Letting k — > +oo gives w^(xQ,t) > Usci(3^o) but the definition of W- implies that it is in fact an 
equality. 

Now if the trajectory is monotone, there are two possibilities: increasing or decreasing. We 
remark that if s i-> Y^^{s) is increasing in l^i, then 9i is necessarily -|-oo and limj_^oo 'W-{xo,t) = 
Usci(^o)- The only remaining possibility is that the trajectory is decreasing, which ends the 
proof. □ 



Now we can proceed with the proof of Proposition 15.31 

Proof of Proposition \5.3l We first examine the case when v{0) < liminf2;_j.o '^^(s^) and apply 
Theorem 13.31 We first remark that, if A) holds for some r] > then it is also true for any fj < r], 
by the Dynamic Programming Principle. Therefore we can assume that rj is as small as we want. 

Then the property u(0) < lim inf^^^o f^i^) implies that, in (j3.10p , necessarily = and Y^^ {rf) = 
if rj is small enough. A simple computation then yields Af(0) > info. /j(0,aj) > An-H(O). 

On the contrary, if B) holds, we know that, for any test function, the minimum is attained at 
X = 0, which implies directly 

Ht{^, v{{)),Duvm = A(?;(0) - ^^^(0)) > . 

Hence, in both cases, v{Q) > u-^lO) which implies the result. 

Since v is Isc, we may now assume in the rest of the proof that v{0) = liminf^-^o ^(2^) = 
lima,j.^o ^(^^fc)- The alternative of Theorem 13.31 can also be applied. In case B), we have exactly as 
above: v{0) > un{0) > min {n^(0), Usci(O), Usc2(0)} . 

It remains to treat case A). For simplicity we assume that, for any k, xt > 0, that is we always 
have i = 1 and we use Lemma 15.41 up to the extraction of subsequences, the following holds 

(i) either, for any x^, we have v{xk) > Usci(2^fc) and we conclude by letting Xk — > 0, which gives 
directly v{0) > Usci(O), since Usci is a continuous function thanks to assumptions [HI] and [H2]; 
(m) or for any Xk, the associated trajectory Y^^ is decreasing. In this case we notice that < 
< Xk and pass to the limit in the expression 



vixk) > w.{xk,t) = / h{Y,\{s),ai{s))e-^'ds + v{Y,\{ei))e 
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which yields 

v(0) > / /i(0,ai(s))e^^'(is + v(0)e"^^i . 

JO 

Iterating the estimate as above we obtain that i;(0) > Usci(O). 

Of course, if the sequence xt hes in 0.2^ we obtain in each case that v{Q) > Usc2(0). Combining 
all the cases above, we get indeed 

v{{)) > min {n^(0),Ugci(0),Usc2(0)} • 

It remains to proove that this min is actually U~(0). First, by comparison in we get 
U"(x) < Usci(a;), i = 1,2. Moreover, (fZT2]) gives U~(0) < uu{0) so that we obtain U~(0) < 
min {ti-^(O), Usci (0), Usc2(0)} . Then, since U" is a supersolution, we can use the reverse inequal- 
ity for V = U~ and conclude that equality holds. □ 

Concerning subsolutions, we have 

Proposition 5.5. Assume [HO], [HI] and [H2]. Let u he a hounded, use subsolution of lil.l\ )- [T73\} - 
[r4\ ). Then 

tx(0) <min{n^S(0),Usci(0),Usc2(0)} = U+(0). 

Proof. We first show the inequality on the left. Notice first that by (f233|) . n(0) < U+(0) < n^^(O). 
Also, by classical comparison arguments, it is clear that in ili, u{x) < Usci(3^)) and the same in ^2 
with Usc2- Hence the inequality indeed holds. Notice that U~^(0) itself also satisfies the inequality 
on the left. 

Now, for the equality on the right the argument is similar to the one we used for the su- 
persolutions. Alternative B) of Theorem 12.71 translates directly into U"'"(0) > li^^(O). In the 
other case, we use Lemma [5.41 and conclude as above that U"'"(0) > Usci(O)- Hence U"'"(0) > 
min {u^^(O), Ugci(O), Usc2(0)}, and combining with the first part of this proof we get equality. □ 

We then turn to the proof of Theorem 15.11 

Proof. We consider the case of U~, the argument being the same for U^. The first part of the 
Theorem has been proved in Proposition 15.31 Then, we solve separately the Dirichlet problems in 
ill and Q2, putting the value of the min as boundary condition at x = 0. We get a solution uj in 
fli U r22 which satisfies u^{0) = U~(0). Hence, by uniqueness for the Dirichlet problem in each fij, 
we end up with = U~. □ 

5.2 State constraint strategies 

We consider dynamics which are given by 

Xxoii) = ai(t) in Qi , X^^it) = 02 (t) in O2 , 

where ai(-),a2(-) € (O, +00; [-1, 1]) are the controls. We are thus in the case where the dynamic 
reduces to bi{x,ai) = a^. the control is actually the velocity of the trajectory. Then we consider 
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the following costs 

li{x, ai) = 1 — ai + e~'^' in f^i , hix, 02) = 1 + a2 + in ■ 

In this case, it is rather obvious that the best strategy for xq > consists in choosing ai = 1, which 
yields a state constraint solution. To be more precise, let us first consider 

Au-^(O) = min <^ /i(2 - ai) + (1 - /i)(2 + 02) : /uai + (1 - /u)a2 = o \ ■ 

IJ,,ai,a2 (. J 

Using the compatibility condition, we can compute the minimum which is atteind for (/x, 01,02) = 
(1/2, 1, -1) and gives u-h{0) = 1/A. 

Now, we can compute the state constraint solution Uusci: for xq > we have 



Uusci(2;o) = inf / f 1 - Qi + 

"i()7o ^ 



dt 



hOO 



e 







hOO 



1 + A 

Indeed, the inf is clearly obtained for the choice ai = 1 which implies that ^^^(t) = xq + t. 
Hence Uusci(O) = 1/(1 + A) < 1/A = n-^(O) for any A > 0. For symmetry reasons, we have also 
Usc2(a;) = Usci(— ic) for x < 0. Then Corollary 15.21 implies uniqueness, so that we conclude that 

for any x G M, 

U-(x) = U+(x) = ^. 
For X > we can also compute the Hamiltonian associated: 

Hi{x,u,p) = sup { — aip + An — (1 — ai + e~'^')} 

QiG[-l;l] 

= sup { — ai(p — 1)} + An — 1 — e~'^' 

QlG[-l;l] 

= |p - 1| + An - e"!""! - 1 , 

and a direct computation shows that U~ is indeed a solution of the HJB equation Hi = in Oi. 
Of course a similar calculus can be done in ^2 which gives H2{x,u,p) = \p + 1\ + Xu — e"'^' — 1, 
for X < 0. 

5.3 "Push-push" strategies 

We now provide an example where the state constraint solutions are not necessarily the best ones. 
Consider 

X^o(t) = ai(t) in $7i , X^^{t) = 02 (t) in $^2 , 
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where ai(-),a2(-) G L~ (O, +00; [-1, 1]) are the controls and the following costs 

li{x, ai) = 1 + ai in f^i , hix, 02) = 1 — a2 in ^^2 j 
Considering the quantity min {n-^(O), Usci(O), Usc2(0)}, it is clear that 

Ati^(O) = min <^ ^(1 + qi) + (1 - /i)(l - 02) : /^ai + (1 - /u)a2 = o \ = , 

fi,ai,a2 L J 

which is attained for the choice {^i, 01,02) = (1/2,— 1,+1). This corresponds to a "push-push" 
strategy, while the state constraint solutions cannot reach the min. Indeed, 



r+co 

Usci(3;o) = inf / {I + ai{t)) e-^' dt 

JO 



hoo 



= inf / {I + X^,{t)) e-^' dt 

ai(-) Jo 

= inf (\ + 

1 



-| -)_oo r+oo , 

X^,{t)e-^' +X X^,{t)e~'^'dt) 
J JO ^ 



which implies Usci(O) > 1/A > 0, which is attained for ai = for xq = (the computation is 
similar for Usc2)- 

Actually, we can compute explicitly U~. We first remark that since li > 0, we have U"*" > 
and U~ > 0. For x E Oi, we choose the control (^, ai, 02) = (1/2, —1, +1) for t > 0. Of course, /.i 
and 02 are not relevant before the first hitting time t{x) = \x\. This strategy consists in reaching 
^ = {x = 0} as fast as possible, then to stay in V. using a "push-push" strategy. With this 
particular choice, 

U"(a;) < / (l + ai(t))e"^*(it+ / /^(x(t), a(t)) e~^* = , 

JO Jx 

SO that U~(x) < 0. Hence we deduce that U~ = 0, since the computation for x < is similar. 
Of course this is an obvious solution of the associated HJB equation which reads in this case (for 
X > 0) 

\ux + 1| + Au = 1 . 

Since the regular strategies give us U~(x) = and U^(x) > the non singular strategies cannot 
be better here, therefore li-H(O) = it^^(O), which implies that uniqueness holds - see Corollary 15.21 
U+ = = is actually the unique solution of (jl.ip . 

5.4 Non-uniqueness and "pull-pull strategies" 

Next we consider the "converse situation", with the same dynamics but now with the costs 
li{x, ai) = 1 — ai + |x| in fii , hix, 02) = 1 -|- 02 + |x| in Q2 , 
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where ai(-),a2(-) G L°°(0, +c5o; [-1, 1]) . Of course, in this example the running cost is not bounded 
because of the |2;|-term but a shght modification would give a similar result. We keep it as it is to 
make simple computations. We have first 

XuniO) = min f/Li(l - ai + |0|) + (1 - + 02 + |0|) : ^ai + (1 - [1)02 = o| = , 

which is attained this time for the "pull-puU" strategy (/U,ai,a2) = (1/2, 1, — !)• 

Notice that if now we consider only the regular trajectories, the "pull-pull" strategies are for- 
bidden. In this case, the restrictions ai < 0, ^2 > imply 

//(I - ai) + (1 - -1- 02) = 1 - fiai + (1 - /u)a2 > 1 . 

Hence we obtain n^^(O) = 1/A, which is attained for ai = Q2 = 0. 
We compute now the state constraint solutions 



Usci(xo) = inf / (1- ai{t)+X^,{t)]e- 
oi()jo V / 

= + inf J / -ai it) e-^* dt + X^, [t]—- + - / X^, (t) e^^* dt 

I JO L -A J A Jo J 



A ai() VA 



+00 



+ inf <; - 1) / ai{t)e-^Ut } . 







For instance, if A = 1 we get the simple solution Usci(3^) = x + 1. More generally, if A > 1, the 
best strategy consists in choosing ai = 1 and the computation gives 

X 1 

Usci(2;) = x^^- 

For symmetry reasons (or a direct calculation) we have 'Usc2ix) = Usci(~2;) for 2; < 0. 

If < A < 1 then we compute as follows, using the hitting time t(xo) and the "pull-pull" 
strategy for t > t{xo) 

Usci(xo) = inf (l - X,,{t) + X.,,{t)]e-^'dt 

= 1 + inf I - [X.„(t)e-^*];(^°) + (1 - A) /^^'°^ X,,(t) e-^* dt\ 
^ "i(-) [ JO J 

= - + rEo+ inf Ul-A) / X,,{t)e-^Ht\. 
A «i( ) ^ Jo ^ 



So, the best strategy here consists in choosing ai = — 1 to reach {x = 0} as fast as possible, then 
to stay there for later times. This gives r(xo) = xq and 



Usci(a^o) = j + xo + {l- X) J {xo-t)e 



XQ 

"dt 



^5±i- 1^(1 -.--.). 
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Hence Usci(O) = 1/A^ > for A > 1, and Usci(O) = 1/A > for A < 1. Here also, by symmetry 
we have Usc2(a;) = Usci(— 3;) for x < 0, so in any case, the "puh-pun" strategy is the best 

niin{ii^(0),Usci(0),Usc2(0)} =n^(0) =0. 

Then if we set 

fUsci(2;) ifx>0, 
lUsc2(2;) ifx<0, 

by Theorem 15.11 we have a subsolution (and actually a solution) in M. But of course this solution 
does not satisfy the condition 

Utt(O) < min{nK(0),Usci(0),Usc2(0)} • 

Notice however that for any A > we always have u^{0) < n^^(O), so that 

7x^(0) = min{n^^°S(0),Usci(0),Usc2(0)} =U+(0). 

Hence by uniqueness in f^i and Q2, we conclude that = U"^. 

In order to prove non- uniqueness let us compute explicitly the minimal solution U~. For x > 
we have, denoting by t{xq) the exit time for the trajectory starting from xq 

\J~{xo)= inf / l-ai(t) + X,„(t) , 

where we are using here the "pull-pull" strategy on 'H = {x = 0} which keeps the trajectory at 
X = for a null cost after t(xo). 

If A < 1, the optimal control is obtained for a = —1 which minimizes at the same time t{xq) 
and the integral multiplied by (1/A — 1). For instance, for A = 1 we get 

U"(xo) = xq+ I dt = l + XQ- e~^^ < njj(xo) = xq + 1 • 

Jo 

Of course the solution is symmetric for xq < 0. In the case A < 1, the explicit solution can be 
computed the same way with a = —1 which gives 

The related Hamiltonian is computed as follows: for x > we have 

Hi{x, u, Ux) = sup I — aiUx + Xu — (l — oi + x) | 

= sup I — ai{ux — l)|+An — X — 1 

= jli^; — 1| + Ati — X — 1 . 

It can be checked that for all A < 1, U~ is indeed the solution of Hi = with U~(0) = 0. Finally, 
since U~ < U"*", uniqueness fails in this situation. 
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6 Approximations, convergence 



Since Problem (|l.ip - (|1.3p - (|1.4p does not have a unique solution, the convergence of approximation 
schemes is not clear in general : indeed some approximations may convergence to U~, some others 
to and we may even have convergence to other solutions. We will consider below two different 
cases. The first one is the case of Filippov's approximations; since it intuitively corresponds to a 
relaxation (where, roughly speaking, a larger set of controls is used), the answer is rather clear: 
we have convergence to the minimal solution U~. On the contrary, we have no general answer 
for the vanishing viscosity method since we have no simple interpretation which may indicate an 
approximation from above or below and therefore a convergence to either U~ or U"*" (see however 
our conjecture below). 

6.1 Filippov's approximation 

A natural approximation of the above problem consists in introducing a continuous, increasing 
function 99 : M — )■ M such that 

lim (p{s) = and lim (p{s) = 1 , 

s— ^— 00 s— >-+oo 

and to study the behavior of the solution : — > M of 

iPs{xN)Hi{x,Ue,DUe) + {1 - (PeixN))H2ix,Us,DUe) = in , (6.1) 

where ipe{xN) '■= ^{xn/^)- 

Contrary to the vanishing viscosity approach (see below), this method keeps record on what is 
happening on the hyperplane by "spreading" it and tracks the controls that fulfill the compatibility 
condition between the two vector fields 6i(x,ai) and 62(2;, 02)- Hence, even the singular strategies 
are taken into account in the limit so that we obtain U~. 

Theorem 6.1. Assume [HO], [HI] and [H2]. There exists a unique Lipschitz continuous solution 
Us of 116. Moreover, as e ^ 0, — )• locally uniformly in M^. 

Proof. The first part of the theorem is clear since the Hamiltonian of Equation (16. ip is coercive: 
by standard arguments, it is straightforward to obtain the existence and uniqueness of the m^'s and 
to prove that they are equibounded and equi-Lipschitz continuous. 

Applying Ascoli's Theorem, we may assume that the sequence {us)e converges locally uniformly 
to a bounded, Lipschitz continuous function u and it is also easy to show that u satisfies (jl.ip - 

In order to conclude, we just have to show that u is also a subsolution of (jl.Sp . Indeed, if this 
is true, the result follows from Corollarv 14.41 (iii). 

Let (j) = (j){y') be a smooth function and let x' be a strict local maximum point of u{y' , 0) — (j){y'). 
We have to prove 

Ht{x,u{x),D'h(^{x'))<^ , 
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where x = (x', 0), i.e. 

-bn{x, a) ■ Dy_4>{x') + \u{x) - lu{x, a) < , 
for any a = (ai,a2,)u) € ^0(2;) for which {^hi{x,ai) + (1 — I-L)h2{x , a2)) -6^ = 0. 

Since 93 is a continuous, increasing function, there exists s € M such that ip{s) = fi. We introduce 
the function ^ 

Usiy) - (t>{y') - -|^ - . 

By standard arguments, this function achieves a local maximum at a point close to x and when 
e — )• 0, we have 

£ e 

Moreover, using that is Lipschitz continuous, the derivative of this term '■= -^{-^—^-^ — s) is 
bounded. 

Now we write the viscosity subsolution inequality 

Ve{{x£)N)Hi{Xe,Us{Xe), Dn(j){x'^) + dNCN) + (1 - (Pe{{Xe)N))H2iXe,Ue{Xe), Dn(j){x'^) + dNCN) < , 
and we notice that, on one hand, 

'Pe{{Xe)N) = ^{s) + o(l) , 

and, on the other hand, the Hi terms are bounded since the gradients are bounded. 
This yields 

ip{s)Hi{Xs,Ue{Xe),Dn(pix'^) + (^TVeAr) + (1 - ^p{s))H2{Xe,Ue{Xe), Dy_4>{x'^) + dNew) < o(l) , 

and, using the form of the ffj's 

ip{s) (-61 (Xe, Qi) • {Dncpix'^) + dNCN) + XUe{Xe) - ll{Xe, Oi)) + 
(1 - (^(s)) (-52(Xe, 02) • {Du(p{x'^) + dNCN) + A'Ue(Xe) - h^Xs, 02)) < o(l) . 

Because of the choice of s and since a = (ai, 02, fJ-) G Aq{x), this inequality is nothing but 

-(/i6l(Xe, ai) + (1 - ti)b2{Xs, 02)) • Du(j){x'^) + Ane(Xe) - (/i/l(x£, Qi) + (1 - /U)/2(Xe, 02)) < o(l) . 

And the conclusion follows by letting e tends to 0. □ 
6.2 Vanishing viscosity approximation 

In this section, we show that the vanishing viscosity approximation may converge to by coming 
back to the example of Subsection 15.41 where non-uniqueness happens because of some singular 
("pull-pull") strategies which give a lower cost. Such strategies are rather instable and it is natural 
to think that, if we add a brownian perturbation, the trajectories will naturally tend to go away 
from X = 0. From the pde viewpoint, this instability is reflected in the fact that the vanishing 
viscosity method does not give U~ in the limit, but U"'". More precisely we have 
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Proposition 6.2. Let us assume that we are in the framework of Subsection \5. 4\ and, for any e > 0, 
consider the solution of the following problem 

- eu'l + H{x,Ue,u'^) = in M, (6.2) 

where H = Hi in 0,i and H2 in Then, as e —?■ 0, the sequence {ue)e converges locally uniformly 
to U+ in R. 

Before proceeding with the proof, let us precise that by a solution Us, we mean a distributional 
solution G W^^^(K). In particular, the possible discontinuity of H when crossing H is not a 
problem in the integrated version of the equation: for any 99 G Cg (K) , 

e / u'^tp' + / H{Xs,Ue,U^)if = . 

Jr jr 

This explains why we will recover in the limit only the strategies already encoded in the equations 
in ill and and not the singular ones. 

Lemma 6.3. For any £ > 0, there exist a unique solution of (j6.2p Us in Wl^'^(]R) for any r > 1, 
which has a at most linear growth. 

Proof. The proof follows classical methods and we are just going to sketch it. For more details, 
we refer the reader to the book of P.L Lions [22] where similar results are obtained. The easiest 
way to prove the existence of is by using first a Filippov-type approximation: in this way, the 
nonlinearity becomes continuous w.r.t. all variables and, since Hi,H2 are Lipschitz continuous 
(therefore at most linear in p), one easily builds a solution which is in Wi;;(M) for all r > 1 and 
which grows at most linearly. We point out that a (uniform in e) linear growth can be obtained by 
remarking that, for K > large enough, zbi(r(|xp + 1)^/^ are respectively sub and supersolutions 

of dOD- □ 

Proof of Proposition \6.SX We notice first that the solution U"*" which is computed in Subsection 
15.41 is always convex. Indeed, this is clear if A > 1 since in this case \J~^{x) = \x\/\ + and a 
straightfoward calculus shows that for A < 1, (U^)"(x) = ^ + (1 — A)e~''*l^l > in the sense of 
distributions so that 

-e(U+)"(x) + H{x, U+(x), (U+)'(x)) = -e(U+)"(x) < . 

Now we consider w := — u^. Substracting the inequations, and using the Lipschitz continuity 
of H(x,u,p) in p (for all u and a.e. in x), there exists a constant C > such that 

-£w" + Xw - C\w'\ <0 in M. 

We first notice that both U+ and Ue grow at most linearly, so does w. If we set := {w — ??(|xp + 
1))+ for some small rj > 0, then Wrj is compactly supported and therefore in VK^'°°(]R). Moreover, 
by similar arguments as in the proof of Lemma 14.31 ^r; still satisfies 

-£w'^ + Xwri - C\w'^\ < in M , 
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at least if e is small enough. 

We consider now a large integer n to be chosen later. By using an approximation of w^""*"^ by 
test functions ^ Cd(I^) ^ N), we can pass to the limit as k ^ +00 in the weak formulation 
and get 

e(2n + l) [ |u;;|2«;2n^^ f ^2n+l^^ _ ^ f |y;^|y;2n,+ l 

Jr Jr Jr 

Using that 

we choose k = 2C/X and obtain 

^ ^ Jr ^ Jr 

Hence if choose n large enough, we get Wj^ = 0. Passing to the limit as ^ we find that w < 0, 
which means U"*" < in M. 

Finally we pass to the limit as e ^ by using the half-relaxed limit method: if 

u{x) := limsup*'U£(x) = limsupUe(y) and u{x) := liminf*Ue(rE) = liminf lie(y) , 

y — )■ X- y —>■ ^ 

£ -> <^ ^ 

then M is a subsolution of (jl.l|) - (|1.3p - (jl.4p and therefore, by Corollary 14.41 u < in M. But, on 
the other hand, U"*" < li^ in M and this gives U"*^ < n < u in M. Therefore U"*" = u = u in M which 
implies the uniform convergence of to U"'". □ 

We notice that the same result holds under the assumptions of Subsection 15.31 but of course in 
this case, U~ = U+. 

6.3 A conjecture 

Another approximation can be used through a combination between the "vanishing viscosity 
method" and the Filippov's method to obtain the approximate problem 

- deAUs + 'feixN)Hi{x,U£,Du£) + (1 - ip£{xN))H2{x,Ue,Du£) = in , (6.3) 

where 6£ is a parameter devoted to tend to zero. Of course, if 5s <C £, the expected behavior of 
is as in the Filippov case, i.e. a convergence to U~ and we think that if (5^ ^ e, then U£ converges 
to U"'" as in the viscous approximation. 
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